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Remarks: 

Applicants have read and considered the Office Action dated July 13, 201 1 and the 
references cited therein. Claims 1 and 30 have been amended. Claims 1-3, 6-13, 15-25, 28-31, 
33, 34, 36-38 and 40-42 are currently pending. Reconsideration is hereby requested. 

Claims 1,3,9-10, 12-13, 15, 17-20, 23-25, 28-31, 37-38 and 41-42 were rejected under 35 
U.S.C. § 103(a) as being unpatentable over Takahashi (US 5,522,789) in view of Suzuki (US 
5,796,427). Apphcants respectfully traverse the rejection. Claims 1 and 30 have been amended 
and patentably distinguish over the combination of Takahashi and Suzuki or any other prior art 
or combination thereof 

The present invention relates to a stereoscopic endoscopic system and method for 
producing stereoscopic color images of an object while compensating for movements of the 
image sensor assembly (e.g., as a result of human operator hand movements). The system of the 
present invention includes an image sensor assembly, a movement detector and a processing 
unit. The processing unit is coupled with the sensor assembly and with the movement detector. 

The sensor assembly acquires a sequence of alternative stereoscopic images of an object. 
For example, the sensor assembly acquires a first right side image of the object (i.e., an image 
acquired from a right side perspective), a first left side image of the object (i.e., an image 
acquired from a left side perspective), a second right side image of the object, a second left side 
image of an object, a third right side image of the object, and so forth (See the paragraph 
beginning on page 45, line 15 and the paragraph beginning on page 48, line 10). The movement 
detector detects movements of the sensor assembly in a plane perpendicular to the optical axis of 
the sensor assembly and produces corresponding movement signals (See page 46, lines 21-29). 

The processing unit receives the detected stereoscopic images from the sensor assembly 
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and receives movement signals from the movement detector, respective of the sensor movements 
(i.e. movement data). The processing unit selects corresponding portions of the alternating 
stereoscopic images for producing a sequence of stereoscopic display images. It is noted that for 
producing stereopsis (i.e., producing depth perception) the display images must show the same 
area of the object from different perspectives. Therefore, the corresponding portions of the 
stereoscopic images are sub-matrices of the stereoscopic images that have the same area size 
(i.e., the same size) and which show the same area of the object. That is, the processing unit 
selects a sub-matrix of each of the detected stereoscopic images, such that all the sub-matrices 
have the same area size and show the same area of the object (See Figures 26-27 and page 
47, lines 1-9, and page 50, line 23-page 51, line 17). 

The sub-matrices are selected for compensating for the movements of the sensor assembly. 
Therefore, the processing unit selects the con-esponding portions of the alternating stereoscopic 
images (i.e., the sub-matrices), according to the received movement data. In particular, the 
processing unit selects each sub-matrix, from its respective stereoscopic image, according to the 
relative displacement of the sensor assembly while acquiring the respective stereoscopic image 
from a point of origin. The point of origin is defined by the average of the movements of the 
sensor assembly. In case the average point of the sensor movements changes and is not constant, 
a new point of origin is defined. Thereby, the processing unit selects a sequence of stereoscopic 
sub-matrices all showing the same area of the object. (See the paragraphs beginning on page 46, 
line 3; page 47, line 1; page 47, line 19; and at page 48, line 1). 

Takahashi 

Takahashi describes a stereoscopic endoscopic imaging apparatus. The system of 
Takahashi includes a pair of relay lens systems, an imagery optical system and a pair of imaging 
devices. The imagery optical system is a verifocal optical system. The system detects 
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movements of elements within the verifocal optical imagery system and accordingly moves the 
image detectors. 

The present invention is directed at producing stereoscopic endoscopic images while 
compensating for movements of the sensor assembly, which are peipendiciilar to the optical axis 
of the sensor assembly. Takahashi fails to teach or even suggest compensating for movements of 
the sensor assembly as a result of, for example, the movements of the hand of the operator. This 
was stated in section 4 (page 3) of the Office Action: 

''Takahashi fails to disclose a movement detector for detecting movements of the 
sensor assembly perpendicular to the optical axis, relative to the object, and 
compensating for detected movements of the stereoscopic sensor assembly ..." 

Therefore, Takahashi cannot be considered as disclosing or even suggesting the present 
invention and merely describes a stereoscopic system. 

Suzuki 

Suzuki describes an image stabilizer as known in the art (See: the background and Figures 
3-4 of Suzuki), including a field memory, a memory controller and a fluctuation detector (See 
Suzuki, column 1, lines 17-25). The fluctuation detector produces information respective of the 
fluctuation of an image pickup device and notifies the memory controller (See Suzuki, column 2, 
lines 3-8). The memory controller identifies an area common to the first and second images on 
the basis of this notification (See Suzuki: column 2, lines 8-16). After the memory controller 
identifies the common area, the memory controller reads only image information representing 
the common area and accordingly produces a new image (i.e., a display image) (See Suzuki, 
column 2, lines 17-24). 

As the common area is smaller than the originally detected images, since a fluctuation was 
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detected by the image pickup device, producing a new image having the same area of the 
originally detected images from less image information. Producing such an image in this manner 
inevitably reduces the image quality. This is shown explicitly in Figure 4 of Suzuki where 
section Al, which represents the area common to the first and second images Fl and F2 
respectively, but which is clearly smaller than either of those images, is used to produce a new 
image F3. Image F3 has the same area as images Fl and F2 but is generated from section Al, 
which is clearly smaller and has less image information. This is the problem of the prior art 
image stabilizer that Suzuki aims to overcome and is stated explicitly in Suzuki as follows: 

" When the common area is identified, the memory controller reads only an image 
information representing the common area . In this case, the memory controller is set 
to have, for example, a long reading period so that a third image may be built by 
means of only this image information (a less amount of information than usual) . 
After that, the image information is converted into a recording signal by the D/A 
converter. 

In the above-mentioned existing image stabilizer, there has been a problem in the 
amount of image information for representing an image to be recorded, although 
compensation of a fluctuation is performed, in case that a fluctuation of an image 
pickup device, namely, a fluctuation of an image happens. In other words, there has 
been a problem that quality of an image represented by a recording signal is 
deteriorated . " (See Suzuki: column 2, lines 17-32, emphasis added ) 

Suzuki's solution to the above problem of the prior art is to supplement the image 
information of the identified common area between a pair of consecutive images, with image 
information taken from the first image of the pair of consecutive images, thereby improving the 
quality of the display image (See Figure 2 of Suzuki and the accompanying description). As 
shown in Figure 2 of Suzuki, the new image Fout includes the portion Al, which was common to 
Fnew and Foid, as well as a portion A2 from Fom, which was supplemented to Fout- 
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Therefore, both the prior art image stabilizer and the Suzuki image stabilizer identify the 
common area without applying any limitations to the identified common area. That is, the 
common area between consecutive images is not necessarily common throughout a sequence of 
consecutive images. For example, the common area between a first image and a second image 
could show a first area of the object while the common area between the second image and a 
third image could show a second area of the object that does not coincide with the first area of 
the object. Additionally, the common area could be of any size and is dependent on the 
movement of the image pickup device between consecutive images. It can be seen that the size 
of the common area may significantly change between consecutive images. Moreover, the 
Suzuki solution of appending image information from a previous image to the common area 
might adversely affect the integrity of the image created from the common area. For example, in 
case the object itself is also moving between consecutive images, the appended area would 
present the object in a first stance while the common area would present the object in a second 
stance, different than the first stance. 

Conversely, the system of the present invention detects a sequence of stereoscopic images 
and selects from the sequence corresponding portions (i.e., sub-matrices) for producing a 
sequence of stereoscopic display images. Thus, as required for producing stereopsis, the selected 
sub-matrices must show the same area of the imaged object and have the same area size. This 
limitation is recited in claim 1 of the present invention as follows: 

^'wherein said processing unit selects correspondins portions of said alternating 
stereoscopic images, according to a signal received from said movement detector 
and compensates for detected movements, thereby producing a visually stable 
sequence of display images, 

and wherein said visually stable sequence of display images comprises a plurality of 
sub-matrices, each one of said sub-matrices is selected from a respective one of said 
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Stereoscopic images, each one of said sub-matrices is located at a distance equal to a 
respective one of said movements from said location of origin, in a direction opposite 
to said respective movement, relative to said location of origin.'' { Emphasis added) 

Therefore, when the average point (i.e., of the sensor movements - the location of origin) 
changes, the system of the present invention determines a new average point. The new average 
point is determined for enabling the system of the present invention to select sub-matrices that 
show the same area of the object and that have the same area size. Otherwise (i.e., without 
determining a new average point), sub-matrices showing the same area of the object would have 
to be smaller. It can be appreciated that as each sub-matrix is located at a distance equal to a 
respective sensor movement from the point of origin, the area size of the sub-matrix decreases as 
the distance from the point of origin increases. As recited in claim 1, the average movements of 
the sensor assembly must be constant to maintain a uniform size for each sub-matrix: 

'' the average of said movements being constant and defining a location of origin 

and wherein said visually stable sequence of display images comprises a plurality of 
sub-matrices, each one of said sub-matrices is selected from a respective one of said 
stereoscopic images, each one of said sub-matrices is located at a distance equal to a 
respective one of said movements from said location of origin, in a direction opposite 
to said respective movement, relative to said location of origin ." ( Emphasis added) 

As detailed herein above, Suzuki carmot be considered as disclosing the present invention 
and in particular as disclosing or even suggesting selecting corresponding portions of the 
detected images as stereoscopic display images (i.e., sub-matrices having the same area and 
showing the same area of the object of the detected images). Therefore Suzuki carmot correct 
the deficiencies of Takahashi. 
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The combination of Takahashi and Suzuki 

As detailed herein above, Takahashi fails to disclose or suggest a method for compensating 
for movements of the sensor assembly (perpendicular to the optical axis of the sensor), and 
Suzuki fails to correct the deficiencies of Takahashi. A combined system of Takahashi and 
Suzuki would include the stereoscopic endoscope of Takahashi complemented by the fluctuation 
compensation system of Suzuki. Such a combined system compensates for movement of the 
image detector of the stereoscopic endoscope between two consecutive images by identifying a 
common image area and producing from that common image area a display image, having the 
area size of the detected images. Additionally, the common image area between a first pair of 
consecutive images does not necessarily show the same area of the object as the common image 
area between a second pair of consecutive images, thereby blurring the stereoscopic display 
images. 

The combined system cannot thus be considered as disclosing the present invention, in 
particular the stereoscopic device recited in claim 1 . In addition, none of the other cited prior art 
publications describe selecting corresponding portions of the detected stereoscopic images as 
stereoscopic display images for compensating for the movements of the sensor. 

The arguments presented herein above with respect to claim 1 apply mutatis mutandis to 
independent claim 30 and Applicants assert that claim 30 patentably distinguishes over the prior 
art for similar reasons. As independent claims 1 and 30 are new, novel and non-obvious over the 
combined teachings of Takahashi and Suzuki (and any other cited prior art or combination 
thereof), the dependent claims also patentably distinguish over the prior art. 

Claims 2 and 33 were rejected under 35 U.S.C. § 103(a) as being unpatentable over 
Takahashi and Suzuki in view of Adelson (US 5,076,687). Adelson fails to remedy the 
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shortcomings of the Takahashi and Suzuki combination. Therefore, Applicants assert that claims 
1 and 30 patentably distinguish over the combination of Takahashi, Suzuki and Adelson. As 
claims 1 and 30 patentably distinguish over the combination of Takahashi, Suzuki and Adelson, 
AppUcants assert that claims 2 and 33 patentably distinguish over the combination for at least the 
same reasons. Applicants therefore request that the rejection under 35 U.S.C. § 103(a) be 
withdrawn. 

Claim 36 was rejected under 35 U.S.C. §103(a) as being unpatentable over Takahashi, 
Suzuki and Adelson in view of Watanabe (US 5,812,187). Watanabe fails to remedy the 
shortcomings of the Takahashi, Suzuki and Adelson combination. Therefore, Applicants assert 
that claims 1 and 30 patentably distinguish over the combination of Takahashi, Suzuki and 
Adelson. As claim 30 patentably distinguish over the combination of Takahashi, Suzuki and 
Adelson, Applicants assert that claim 36 patentably distinguishes over the combination for at 
least the same reasons. Applicants therefore request that the rejection under 35 U.S.C. § 103(a) 
be withdrawn. 

Claims 6, 8, 1 1, 16, 21-22 and 40 were rejected under 35 U.S.C. §103(a) as being 
unpatentable over Takahashi and Suzuki in view of Watanabe. Watanabe fails to remedy the 
shortcomings of the Takahashi and Suzuki combination. Therefore, Applicants assert that claims 
1 and 30 patentably distinguish over the combination of Takahashi, Suzuki and Watanabe. As 
claims 1 and 30 patentably distinguish over the combination of Takahashi, Suzuki and 
Watanabe, Applicants assert that claims 6, 8, 1 1, 16, 21-22 and 40 patentably distinguish over 
the combination for at least the same reasons. Applicants therefore request that the rejection 
under 35 U.S.C. §103(a) be withdrawn. 
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A speedy and favorable action in the form of a Notice of Allowance is hereby solicited. 
If the Examiner feels that a telephone interview may be helpful in this matter, please contact 
Applicants' representative at (612) 336-4728. 

Please consider this a PETITION FOR EXTENSION OF TIME for a sufficient number 
of months to enter these papers or any future reply, if appropriate. Please charge any additional 
fees or credit overpayment to Deposit Account No. 13-2725. 
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Respectfully submitted. 



MERCHANT & GOULD P.C. 



Dated: 




G?^g(5ry A. Sebald 
-^Keg. No. 33,280 
GAS/km 
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